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ABSTRACT: We present our efforts towards building an observational system for measuring 
classroom activity. The goal is to explore visual cues which can be acquired with a system of video 
cameras and automatically processed to enrich the teacher's perception of the audience. The 
paper will give a brief overview of our methodology, explored features, and current findings. 
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1. INTRODUCTION 

As much as humans developed as social animals, our perceptional and cognitive abilities still limit the 
comprehension of our interlocutor(s). While we are capable of interpreting unspoken signals in a one-to- 
one conversation, the task becomes much more challenging when faced with an audience of thirty or 
more students, with minimal interaction. We aim to enrich the classroom ecosystem with information 
about the state of the audience's attention, presented to the teacher to raise the awareness of how the 
students are receiving the lecture. The research is rooted in concepts of observational data gathering, 
meaning that our main source of information is not based on probing the information from the students 
(e.g., using an application to state their attention), but rather on analyzing the cues we can gather from 
body language to deduce the state of the audience. 

The data is captured with a system of cameras positioned around the blackboard of the classroom. The 
video material is processed with Computer Vision (CV) techniques such as motion detection, head 
detection, and head orientation estimation. Our current efforts are aimed at determining meaningful 
patterns of behaviour and finding the most salient features. The second part of the research aims to 
model student behaviour and finally to discuss how to present the information back to the teacher in 
order to encourage reflective practice in teaching. 

Our efforts build on the concepts of classroom orchestration (Dillenbourg & Jermann, 2010) and teacher¬ 
centric structure of the classroom. The main difference from the current approaches, such as the usage 
of clickers (Caldwell, 2007), is the principle of unobtrusiveness (Webb, Campbell, Schwartz, & Sechrest, 
1999), which constrains our interventions so that they will enrich the teaching process without imposing 
a specific format of the class or adding another feedback tool to the classroom. 
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Figure 1: a) Motion vectors visualized on top of the students. Individual vectors are grouped into motion 
paths depicted as lines of different colours, b) Head detections and estimated gaze directions for a group 
of students in the classroom. 


2. METHODOLOGY 


We conducted a series of experiments for capturing student behaviour during class. Given the 
complexity of the classroom ecosystem, we conducted a holistic analysis of the classroom (Raca & 
Dillenbourg, 2014), capturing many aspects of classroom life including: 

• Videos of student behaviour from which we extracted motion intensity and head 
orientation for each student over time (shown in Figure la and lb). 

• Video of teacher actions/lecture slides: we applied tracking algorithms to find the position 
of the teacher over time and we annotated events during the lecture (slide changes, 
question/answer periods). 

• Questionnaires: we distributed questionnaires on which the students reported their level of 
attention, perception of overall classroom attention, material importance, and teacher 
energy. 

• Interviews: a smaller subset of students participated in the interview sessions in order to 
evaluate student perceptions of the system and validity of the questionnaires. 

• Eye-tracker: the teachers wore a portable eye-tracker during the lectures, from which we 
extracted the gaze patterns of the teachers. 


3. RESULTS 

Our initial set of experiments on motion showed no direct connection between the motion of the 
students and their attention levels (reported in the questionnaires), but gave grounds for the definition 
of the concept of "motion lag" (Raca, Tormey, & Dillenbourg, 2014) — the idea that high-attention 
students will more likely be synchronized in their actions. From this, we hypothesize that the 
synchronization influence comes from the external environment, and that the dominant signal in a good 
classroom will be the teacher. We also observed that the spatial arrangement is also manifested in the 
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results, neighbouring students being more likely to synchronize and that the intensity of the motion 
decreases with the distance from the teacher (Raca, Tormey, & Dillenbourg, 2013). 

Our next steps are directed towards refining our head-pose estimations (Raca & Dillenbourg, 2014) and 
analyzing the observations received from this data stream before merging them into a unified model 
with earlier findings. 
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